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I. AIRCREW COORDINATION TRAINING 


A. STATEMENT OF THE OBJECTIVE 

The purpose of this report is to measure attitudinal change resulting from 
aircrew coordination training. Attitudinal change is necessary for the 
ultimate goal of this training program to be achieved. A training program 
with concepts and theories adequate to cause attitude change, should result 
in behavior change because attitudes and behavior are assumed to be linked 
(Helmreich, Foushee, Benson, Russini, 1986). Behavior more conducive to 
safe operation of aircraft is the ultimate goal of Aircrew Coordination 
Training. 

This thesis examines the attitude changes occurring in P-3 Orion aircrew 
personnel who have attended training administered by Patrol Squadron 
Thirty-One. The Cockpit Management Attitudes Questionnaire (Helmreich, 
Wilhelm, and Gregorich, 1988) was administered to seminar participants 
prior to and immediately after completion of training. Changes in the 
responses of survey participants may be indicative of changes in attitudes 
caused by training (Helmreich, Foushee, Benson, Russini, 1986). The 
consistency and magnitude of changes may provide patterns of change 
indicative of aircrew coordination training effectiveness or ineffectiveness. 
This report will attempt to identify and explain the significance of survey 


response changes. 


B. BACKGROUND 
1. Aircrew Coordination Training Origins 

There has always been an inherent element of risk associated with 
the flight of aircraft. As advancements in flight operations have occurred, the 
element of risk has been sufficiently reduced to make aircraft operations 
practical for a multitude of purposes. Significant improvements in aircraft 
design, air traffic control and weather forecasting have reduced the danger of 
flight operations. 

Significant events in the development of crew coordination training 
are detailed by Manningham (1986) starting with events during the early 
1970s. The term “pilot error” was an often used description applied to explain 
many mishaps. Researchers at the National Aeronautics and Space 
Administration conducted interviews and observed the performance of crews 
to further understand the cause of “pilot error.” Investigations into the 1972 
crash of an airliner into the Florida everglades, a 1985 crash of a Lockheed 
Electra soon after takeoff in Nevada and other mishaps identified the failures 
resulting from poor crew coordination. The severity and complexity of crew 
coordination error was recognized by the National Transportation and Safety 
board and the Federal Aviation Administration. The need for a new 
approach to crew training was identified. To meet this need individuals from 
all sectors of aviation have sought to understand crew coordination error. 
The results of approximately fifteen years of research have been the 
development of crew coordination training programs that are now a part of 


almost every aviation training program. 


Human factors research has determined that pilot experience, age and 
time of day are three important variables in aviation safety. Research has 
identified the "Most Dangerous Pilot" (Borowsky 1990) as one who has only 
five hundred hours or less experience in the assigned aircraft and less than 
one thousand hours total. A newly qualified P-3 aircraft commander in many 
cases fits this description, and is supported by second and third pilots who 
have even less experience. As pilots age they become less likely to make the 
skill errors common to junior pilots but become more likely to make 
procedural errors. Familiarity tends to cause complacency in more senior 
Navy pilots. Further, a dramatic increase in the number of mishaps per 
100,000 flight hours occurs as pilots fly during the 1800-2400 hour time frame. 
During night time hours, the mishap rate increases from 2 to 4.2 mishaps per 
100,000 flight hours(Borowsky, 1990). Inexperience, age and time of day are 
important and known tendencies. 

There is yet another human factor that is more significant than 
inexperience, age or time of day. At present, the most significant causal factor 
in aviation mishaps is crew coordination error. Aircrew error accounted for 
55 percent of the overall Navy/Marine mishap rate during the years 1985 
through 1990 (Borowsky,1990). The cost of aircrew error during this time was 
245 invaluable lives and 187 aircraft worth 1.7 billion dollars (Borowsky, 


£290)». 


!Written permission from the Commander Naval Safety Center was obtained to allow 
utilization of Naval Aviation mishap data. 


The need for extensively trained personnel to operate aircraft is 
recognized by civilian and military aviation authorities. Training programs 
to date have emphasized thorough knowledge of aircraft systems, and flight 
procedures. However, reconstruction of past aviation mishaps has identified 
human failures not addressed by traditional training programs. Fully 
qualified and extensively trained personnel have become involved in 
mishaps due to human error. The nature of this other category of human 
error has been identified and is the target of new training programs in 
civilian and military aviation. The field of study supporting the research and 
training is widely known as Cockpit Resources Management. Recognizing 
the important role of crewmembers other than cockpit personnel, the Navy 
has entitled its program Aircrew Coordination Training (ACT). 

2. Aircrew Coordination Training Development 

Interest in ACT research and training has developed because human 
error is the most significant cause of aviation mishaps. The type of errors that 
are the focus of ACT include errors of decision-making, judgment, and 
communications [Cooper, White, Lauber, 1980]. Analysis of cockpit simulator 
training demonstrates that crews who more effectively utilize crew 
coordination concepts, are more successful when coping with in-flight 
emergencies [Foushee and Manos, 1981]. Dr. John K. Lauber of the National 


transportation Safety Board defines ACT as: 


The effective utilization of all available resources—hardware, software, and 
liveware—to achieve safe, efficient flight operations (Lauber, 1986). 


Organizations currently devoting resources to ACT research and 


implementation include the Federal Aviation Administration, National 


Aviation and Space Administration, National Transportation Safety Board, 
commercial airlines, military aviation and universities. Improvements in 
crew coordination are the focus of so many groups because of the necessity for 
improvement. 

ACT Training programs vary widely in content and method of 
implementation. Significant research into ACT has produced a theoretical 
base too large for a single program to cover. ACT training programs must 
choose the most appropriate material considering resources available to 
implement a program and the mission of the aircraft. A resource intensive 
method of implementation includes the use of flight simulators. Via 
observation of simulator events the practicality and need for ACT was first 
demonstrated. Simulator training is highly useful for decision-making, 
communication and coordination training [Foushee and Manos 1981]. The 
alternative to simulator training is classroom instruction. The P-3 ACT 
course utilizes classroom instruction combined with role play exercises to 
demonstrate ACT concepts. Future P-3 ACT courses may incorporate the 
utilization of simulators if the resource is made available. 

3. The Aircraft 

The Lockheed P-3 Orion aircraft has been utilized by aircrews of the 
United States Navy since 1962 for maritime patrol operations (Figure 1.1). 
The primary mission of the aircraft is anti-submarine warfare. Squadrons of 
P-3 aircraft are deployed throughout the world on a continuous basis, 
monitoring maritime activities of other nations. The aircraft is capable of 
transoceanic flights in excess of fifteen hours (P-3C Flight Manual, 1983). The 


current version of the aircraft is operated by a crew of twelve including: three 


pilots, two flight engineers, two acoustic equipment operators, a tactical 
coordinator, navigator, radar operator, in-flight technician and 
ordnanceman. The twelve members form a combat air crew. To allow 
improved teamwork and efficiency, combat aircrews conduct all training and 
operational missions as an intact twelve-member group. The crew is 
responsible for operation of an extensive variety of electronics equipment 
designed to allow detection, tracking and attack of submarines and ships. The 
aircraft is capable of carrying mines, torpedoes, anti-ship missiles and gravity 


bombs to carry out wartime missions. 
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Figure 1.1. United States Navy P-3C Orion 


4. P-3 Orion Mishap History 
The P-3 Orion began operation in the U.S. Navy fleet in 1962. Since 
that time forward deployed squadrons have been patrolling the waters of the 


Atlantic, Indian, Pacific and Mediterranean oceans. During the twenty-nine 


years of operation thirty-four aircraft have been destroyed and 247 lives have 
been lost as a result of mishaps [(Hess, 1983), (Borowsky, 1992)]*. Despite the 
large loss of life the P-3 is considered safe relative to other aircraft in naval 
aviation. For the years 1985-1991 the F/A-18 mishap rate per 100,000 flight 
hours was 4.23, whereas the P-3 mishap rate was .19 (Borowsky, 1991). Of the 
thirty-four P-3 aircraft destroyed, twenty were destroyed while at forward 
deployed locations. Figure 1.2 shows that eight of the thirty four mishaps 
were known to be caused by mechanical error, twenty-four were in part 
caused by aircrew error and two were due to unknown causes (Hess, 1983, 


Borowsky 1992). 


2 Aircraft destroyed were involved in Class A mishaps. OPNAYV instruction 3750.6Q 
defines a Class A mishap an event where total damage is $1,000,000 or greater; or the aircraft 
is destroyed or missing; or a fatality or permanent total disability occurs with direct 
involvement of Naval aircraft. 


HM AIRCREW ERROR 
YA MECHANICAL ERROR 
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Figure 1.2. Causes of P-3 Class A Mishaps 1963-1992 


II. P-3 AIRCREW COORDINATION TRAINING 


A. TRAINING PROGRAM IMPLEMENTATION 

As authorized by Commander Patrol Wings Pacific, Patrol Squadron 
Thirty-One was tasked to provide Aircrew Coordination Training for all P-3 
squadrons based at Naval Air Station Moffett Field and Naval Air Station 
Memeers FOint. Patro!’Squadron Thirty-One is the west coast P-3 fleet 
replacement training squadron. Under the direction of the command 
training specialist, Dr. Henry H. Smith, the squadron first trained its own 
group of facilitators. Utilizing the command training specialist plus a cadre 
of its own instructor pilots and flight officers, Patrol Squadron Thirty-One 
completed seminars at each squadron. The initial seminars were followed by 
the training of facilitators from each squadron who then completed the task 


of training all twelve combat aircrews. 


B. SEMINAR DESIGN 

The P-3 Orion Aircrew Coordination Training seminar is designed to be 
facilitated by two squadron instructors who guide a combat aircrew through 
an eight-hour progression of concepts and ideas relevant to flight safety and 
efficient crew coordination (VP-31 ACT Facilitator Guide, 1989). Key to the 
success of the program is the concept of allowing each squadron the 
opportunity to select its own instructors. Internally selected instructors are 
more easily accepted and are more capable of understanding the perspectives 


of trainees. Aircrew Coordination Training Instructors utilize a facilitator 


guide, instructional guide and flip-chart and video presentations during the 
day-long course. Ideally, the setting for the seminar is a relaxed atmosphere 
wherein participants wear flight suits instead of uniforms, form a circular 
seating arrangement for ease of conversation and are encouraged to offer 
their views on the concepts at any point during the seminar. A full eight 
hours of discussion is usually required to complete the seminar, including a 
working lunch wherein members are encouraged to order out pizza or 
sandwiches for the group. The setting for the seminar, ideally, is away from 
the squadron so that interruptions are kept to a minimum allowing 
crewmembers to focus on the material presented. Critical to the success of the 
program is the promise of confidentiality to crewmembers who share 
personal information with the group. Incidents that have nearly resulted in 
aviation mishaps eventually happen to all crewmembers and provide 
valuable material for the seminar. The entire discussion is confidential 
among the crew and the facilitators. Only the promise of confidentiality will 


allow open and frank discussion of ideas and experiences. 


C COURSE ORIGINS AND CONTENT 

The field of Cockpit Resources Management, includes a broad base of 
knowledge put together over several decades of research and application. A 
variety of Cockpit Resources Management programs are being presented 
world wide to aircrews of civil and military aircraft. Each program is 
different, as the needs of the organization or the mission of the aircraft dictate 


the content of the course. 
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The P-3 ACT course is a version of the program put together by three Air 
Force Reserve pilots. Lt.Col. Biegalski, Lt.Col. Halliday Major Houle and 
Major Inzana designed and implemented an ACT course for crews flying the 
C-5 transport aircraft (Halliday, Biegalski, Inzana, 1986). The course structure 
is guided by the concept that the goal of ACT is to provide the aircrew with 
something immediately useful and practical. Videos and taped role plays are 
used to demonstrate ACT concepts and to provide feedback. Like the P-3 
version of their course, the C-5 course facilitators guide a day-long discussion 
in a classroom environment. The link between Air Force ACT and P-3 ACT, 
is Dr. Henry H. Smith who modified, tested and implemented a revised 
version of the Air Force course. With the support of Patrol Squadron Thirty- 
One Commanders Alford, Bozin and Hull, and Dr. Smith integrated ACT 
into the VP-31 fleet squadron training program. 

iivescontent of the course offered to the aircrews of the United States 
Navy P-3 Orions is structured to suit the unique features and mission of the 
airplane they fly. There are five main sections to the P-3 aircrew coordination 
training course (VP-31 ACT Instructor Guide, 1990). The five main sections 


include: 
e Synergy 
¢ Team Creation/ Building 
e Style of Personality 
¢ Communication 


e Decision Review 
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D. OBJECTIVES AND METHODS OF P-3 ACT 
Each section included in the P-3 aircrew coordination training course is 
designed to achieve specific objectives critical to aircraft safety. The objectives 
expand and emphasize concepts participants are familiar with in practice but 
lacking in understanding and application. The course material emphasizes 
application of the objectives through education, demonstration and role play 
activities. 
1. Synergy 

synergy occurs when the whole of an aircrews’ effort is greater than 
the sum of its parts. Synergy occurs when crewmembers work together as a 
team. The P-3 Aircrew Coordination Instructor Guide indicates that synergy 
will only occur when all share a joint and co-equal sense of responsibility for 
the safe completion of a flight. Lack of synergy has been responsible for many 
previous aviation mishaps wherein a crewmember fails to volunteer or 
accept information necessary to prevent a mishap. 

synergy is difficult to achieve because of conditioning that leads to 
withholding information critical of other crewmembers. Information is 
withheld because of barriers between crewmembers. For example, in the 
military relative rank can create a barrier. Junior ranking personne! are 
reluctant to correct the mistakes of senior ranking personnel. Lack of 
experience can be intimidating for a recently trained crewmember. 
Inexperienced personnel have failed to offer valid information because they 
lacked confidence in their abilities or were intimidated by a senior person 


(VP-31 ACT Instructor Guide, 1990). 


We 


For synergy to be achieved all crewmembers must recognize the 
responsibility of offering their observations to others. All crewmembers must 
be receptive to information offered to them. This requires that barriers to the 
transfer of information be sufficiently reduced for information to be 
exchanged, no matter what the rank or experience level involved. 

synergy is demonstrated to seminar participants through a general 
knowledge quiz. Subsequent to the quiz individual scores are tallied. 
Individuals normally score in the range of seven to ten correct scores out of 
twenty. The group is then allowed to work together on the quiz. Group 
scores tend to significantly exceed the score of any individual. Groups 
regularly come up with eighteen to twenty correct answers on the quiz, 
demonstrating the effect of synergy. 

2. Team Creation/Building 

Richard Hackman of Harvard University, developed an aircrew 
behavior model titled the Life Cycle of the Crew. The model explains 
elements of crew behavior critical to safe flight operations (Hackman, 1989). 
During the course of a flight the crew will progress through several phases of 
team development. Crews proceed through the phases of team development 
with differing degrees of effectiveness. Dr. Hackman identified the following 


four distinct phases of the Life Cycle of the Crew: 
e Team Creation 
2 eam Building 
¢ Team Acting/Execution 


e Team Termination 
The team creation phase occurs when members of a flight crew first 


meet to discuss a flight. The team creation phase only lasts for five to ten 
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minutes. During this time the flight crew leadership, a Pilot or Flight Officer, 
establishes the behavioral norms as related to safety, communication and 
cooperation. Hackman observed that the norms established at the initial 
briefing usually become fixed for the remainder of the flight. 

To effectively emphasize the importance of safety, the initial team 
creation phase must establish the expected level of regard for safety. 
Crewmembers should be encouraged to avoid unsafe practices and look for 
hazardous situations that could affect others. For effective communication to 
occur on board the aircraft, barriers to communication must be overcome. 
Barriers to communication include: superior/subordinate relationships, 
fatigue, distractions, noise, workload, instructor/student relationships and 
communication by interphone. Crewmembers should be encouraged to 
overcome the barriers to communication when important messages must be 
conveyed. The importance of cooperation in the role of mission success 
should be established from the outset of the briefing. Cooperation requires 
that crewmembers assist others when their work is completed. Safety, 
communication and cooperation are key elements in the Life Cycle of the 
Crew. 

To illustrate concepts defined by Hackman, a video presentation of a 
briefing emphasizing the utilization of these concepts is shown to seminar 
participants. The video includes a twelve-member P-3 crew conducting an 
initial briefing for a surface ship surveillance flight. Following the video 
tape, participants are given a hypothetical mission assignment and asked to 
conduct their own briefing. After completion of their mission brief, 


crewmembers view a video playback to observe their ability to include safety, 
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communication and cooperation as part of a briefing. The next three phases 
in the Life Cycle of the Crew are addressed by a discussion with seminar 
participants led by the facilitators. 

Team building occurs during spare moments before and during 
flight. Aircrew who choose to interact during this time are better able to 
communicate during the mission phase. Team building communication 
addresses events that will affect the crew including: the addition of a new 
member, destination billeting, inflight meals and other factors affecting a 
crewmember's performance. The team Acting/Execution phase occurs 
during flight when crewmembers perform their duties. This is when the 
ability of the crew to handle safety and mission related activities is carried out. 
The team termination phase occurs after mission completion when 
crewmembers recapitulate events. During team termination, crew 
coordination successes or failures should be identified and lessons learned 
from them. 

3. Style of Personality 

Personalities of individuals on a crew can affect quality of interaction 
among crewmembers. For synergy to exist, crewmembers should have 
knowledge of differing styles of personalities. Four distinct styles of 
personalities are compiled into a behavior model for the seminar. Distinctive 
Styles are given names to allow easy association and understanding. The 
styles of personality discussed are the driver, analytical, amiable and 
expressive. The driver is said to be: industrious, systematic, persistent, detail 
oriented, serious, exacting and precise. The analytical person is characterized 


as: objective, determined, requiring, independent, pragmatic, efficient and 


ie 


decisive. Amiable personalities are: supportive, loyal, friendly, responsive, 
dependent, easy going and cooperative. The expressive personality 1s: 
imaginative, out-going, stimulating, enthusiastic, spontaneous, fun-loving 
and ambitious. 

Facilitators lead the discussion and explain to crewmembers how 
personality can affect achievement of synergy. No attempt to modify 
personalities is done by the course. Personalities are assumed to be fixed 
(Gregorich, Helmreich, Wilhelm, Chidester, 1989). Crewmembers must learn 
to adapt to each other to promote synergy. Each personality style can affect 
the crew in positive and negative ways. Effective synergy is achieved when a 
crew can combine the best attributes of each personality. The discussion of 
personalities is intended to promote the awareness of personality as a major 
factor in crew interaction and illustrate how crewmembers can adapt to each 
other. Through education a crew should be more able to cope with the four 
personality groups identified in the seminar. 

4. Communication 
The third model introduced is the cornerstone of the seminar. The 


model is named the Synergy Formula and it is composed of three parts: 
¢ Questioning 
* Promoting 


o™ Connlict 

These concepts are the critical elements in the communication 
process that affect a crew's ability to cope with challenging flight scenarios. 
When circumstances occur that could result in development of an unsafe 


situation, crewmembers should question what is occurring, promote their 
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concerns about action taken and seek to resolve conflict. Facilitators explain 
the concepts and give practical examples of their utilization. 

Questioning, in the context of aircrew coordination training, is an 
attitude that all crewmembers should possess. Each crewmember should 
always be aware of the current phase of flight and plan of action. By 
constantly checking one's actions and cross checking actions of others, 
mistakes can be detected early and corrective action taken. Facilitators 
encourage junior personnel to cross check actions of more experienced 
crewmembers. Senior crewmembers are encouraged not to feel threatened by 
a junior crewmember questioning their actions. Questioning requires that 
crewmembers learn to doubt the status quo is correct and accept that everyone 
is capable of making mistakes. 

Promoting is recognition and encouragement for crewmembers to be 
assertive when necessary. A group interaction decision-making phenomena, 
group think, can cause inferior decisions to be made by the group (Janis, 1972). 
To avoid group think behavior, an individual who believes his/her opinion 
is correct and that the group is wrong should voice that opinion until a 
Satisfactory answer is provided. Facilitators explain promoting as a 
responsibility inherent in assignment to an aircrew. Crewmembers are not 
qualified until they have demonstrated a satisfactory level of competence. A 
qualified crewmember is responsible for being assertive and expressing 
concerns not addressed by the present course of action. Qualified 
crewmembers should not tolerate a situation if they are not satisfied with the 


course of action and have not received a satisfactory answer to their concerns. 
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Conflict is a desirable element and necessary for optimum crew 
coordination. Crewmembers must learn to resist tendencies to avoid conflict 
out of respect for rank or experience. Conflict is a natural result of 
questioning and promoting. Conflict occurs as crewmembers approach 
Situations with different values, experience, training, opinions and 
personalities. A crew able to effectively utilize conflict can realize benefits. 
Ideally, conflict will lead to better problem solving, deeper thinking and a 
more effective decision making process. 

5. Decision Review 

Decision Review is the method used to put synergy into effect. 
Decision Review is also known as a Bubba Review. By utilizing a word not 
popular in present vernacular, the Bubba review is immediately associated 
with the synergy model. A crewmember who identifies an unsafe situation 
developing should demand a Bubba Review. During a Bubba Review, 
questions causing concern are promoted to other crewmembers. A 
crewmember requesting a Bubba Review believes an unsafe situation has 
developed. Other crewmembers should attempt to resolve the conflict that 
has occurred by reexamining the present course of action. Through 
resolution of the conflict the current course of action can be continued or an 
alternative implemented. 

A bubba review can be demanded at any time oy any crewmember. A 


bubba review is composed of three questions: 
e Sir, what 1s the decision? 
¢ What are the consequences of the decision? 


e Is there a new, alternative, better decision? 
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A bubba review causes a more open and complete decision making 
process. The completion of the review puts relevant issues in the open and 
allows a decision to be made based on issues identified by everyone. Another 
Bubba Review benefit is the effect of informing everyone on the aircraft the 
current course of action. Seminar participants are encouraged to write the 
synergy formula on one side of a note card and the three Bubba Review 
questions on the other side. The card should be kept in the individual's 


inflight checklist for immediate reference. 


E. ACT EFFECTIVENESS CRITERIA 

The driving force for widespread implementation of ACT and 
justification for its continued development is preservation of lives and assets. 
The criteria for determining if that goal is occurring are not easily measured. 
Number of mishaps per one hundred thousand flight hours, is the most 
popular criteria used in assessing effectiveness of Naval Aviation safety. For 
the year 1991 the mishap rate for Naval Aviation was 2.91 per 100,000 flight 
hours (Borowsky, 1990). Statistically, that number is very low relative to the 
number of hours flown. Since mishaps for some aircraft are infrequent 
events, measurement of mishaps per hours flown can be greatly affected by 
chance. A few chance incidents can greatly affect the mishap rate (Helmreich, 
Chidester, Foushee, Gregorich, Wilhelm 1989). A mid air collision of two 
aircraft or the crash of a formation flight define a single event that would 
Skew the mishap rate figure. Only by observing mishap rates over a long 
period of years could any distinct trend be determined. Claims of immediate 


reductions in mishap rates via ACT may be premature. Since organizations 
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cannot continue a program such as ACT for an extended period on a trial 
basis, short term measures of program effectiveness are necessary for 
justification and further implementation. 

To provide more immediate and valid measures of ACT effectiveness, 
Helmreich, Foushee, and Wilheim (1989) have developed methods of 
assessment that can be applied. Instead of one measure, multiple measures 


are recommended. Three indicators are suggested including: 
¢ Outcome Measures 
e Process Measures 


¢ Moderator Factors 

The implementation of ACT should have observable effects. These 
effects are called outcome measures and include incidents where ACT 
training prevented a mishap, changes in crew coordination attitudes /abilities 
and a favorable response by crew members towards ACT (Helmreich, 
Foushee, and Wilhelm, 1989). Process measures are actual changes in 
execution briefing, preflight, flight and postflight evolution caused by 
implementation of ACT procedures. Process measures represent the 
implementation of ACT concepts into normal operating procedures. 
Moderator factors are composed of human and material resources and 
include organizational support and environmental factors that affect 
implementation of ACT. The implementation of ACT demands change from 
those involved. To overcome the resistance to change inherent in large 
organizations, moderator factors must support ACT. Success or failure of an 


ACT program can be determined by the moderator factors. 
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F. COCKPIT MANAGEMENT ATTITUDES QUESTIONNAIRE 

The data set utilized to provide an analytical basis for this report is 
derived from the Cockpit Management Attitudes Questionnaire [(CMAQ), 
(Helmreich, Wilhelm, Gregorich, 1988), (Appendix A)]. Based on a study of 
60 aviation mishaps occurring from 1968-1976 (Cooper, Lauber, 1980) five 


common errors related to crew coordination were identified including: 
e Preoccupation with minor technical problems 
¢ Inadequate leadership 
e¢ Failure to delegate tasks and assign responsibilities 
e Failure to set priorities 


e Failure to communicate intent and plans 

The crew coordination attitudes necessary to prevent a reoccurrence of 
these failures were determined. The attitudes were compiled into a survey 
with a corresponding scale to allow the degree of concurrence of disagreement 
expressed by a respondent. The survey was titled The Cockpit Management 
Attitudes Questionnaire (CMAQ). An updated format is widely used for civil 
and military analysis of ACT effectiveness (Gregorich, Helmreich, Wilhelm, 
1989). Responses to CMAQ survey questions can be analyzed by statistical 
methods to determine effects ACT training. The degree of concurrence or 
disagreement with survey questions indicates disposition towards attitudes 
critical to flight safety. 

Trends can be expected in responses to the CMAQ. Organizations whose 
aircraft and mission are conducive to ACT implementation score more 
favorably to survey responses. For multi-crewed aircraft (P-3), safety and 
mission effectiveness are positively correlated to crew coordination ability. 


For single piloted aircraft (F-18), the potential benefit of crew coordination 1s 
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less as safety and mission effectiveness are often dependent upon one 
individual. This does not imply that pilots of single seat aircraft cannot 
benefit from ACT. Often pilots of single seat aircraft work in coordination 
with other aircraft and personnel external to their platform. History effects 
explain that aircrew members who have long term exposure to ACT respond 
differently than those who don't. Exposure to formal and informal training 
methods eventually causes increased awareness and acceptance of ACT 
(Gregorich, Helmreich, Wilhelm 1990). 

The survey was modified for P-3 ACT so that question terminology 
would be compatible with P-3 aircrews. For example, instead of referring to 
the first pilot and load master the P-3 CMAQ version referred to the Plane 
Commander and Flight Engineer. Modification of question terminology was 
done so that original concepts remained intact. The P-3 seminar is not 
designed to teach directly to the CMAQ survey. Some CMAQ concepts are 
directly covered by the P-3 seminar, others are indirectly addressed and some 
are not reviewed at all. Only the portion of the survey designed to measure 
attitudes, is addressed in this thesis. 

The ability to change attitudes is an important measure of the 
effectiveness of the P-3 ACT seminar. The CMAQ has been widely used in 
aviation training as a valid measure of attitudes affecting crew coordination. 
Policy makers can use this CMAQ analysis to guide the future of P-3 ACT. 
This thesis contributes to the knowledge necessary to direct the future of P-3 
ACT. 

The fit of ACT as a valuable part of a much larger aviation safety 


program, can be measured by CMAQ results. Failure of P-3 ACT to provide 
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useful training would indicate the need for a different approach to imparting 
crew coordination knowledge. Success of P-3 ACT would indicate that the 
present seminar format is accomplishing the objective of changing attitudes. 
An outcome between the two extremes would indicate a need for refining 
current efforts. 

Crew coordination training programs vary according to the needs of an 
Organization and its users. Concepts and methods of instruction can be 
varied to meet the needs of a group. The P-3's diverse group of crewmembers 
presents a challenge in developing a course appropriate for all 
crewmembers. Crewmembers differ by aircrew position, P-3 flight hours, 
years military, age and other variables. The degree to which the challenge of 
diversity is addressed can be measured by the uniformity of CMAQ responses. 

If P-3 ACT is designed considering aircrew diversity, CMAQ responses 
will be similar across the groups. If P-3 ACT is less effective for particular sub- 
groups they will respond differently from the norm. To address 
requirements of a group that responds differently to ACT many options exist. 
The course could be amended for the entire group. The course could be 
amended and taught separately for the group with different responses. Lastly, 
ACT could be discontinued for the group responding below the norm. 
Ideally, all groups respond the same regardless of their crew position or 


military background. 
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G. P-3 AIRCREW COORDINATION TRAINING 

This thesis does not represent the first research effort into P-3 Aircrew 
Coordination Training. John Wilhelm and Robert Helmreich of the 
University of Texas at Austin prepared a report as part of the NASA- 
University of Texas cooperative inquiry into Cockpit Research Management 
Training (Wilhelm, Helmreich, 1990). One aspect of their report, published 
in March of 1990, analyzed crewmembers’ feelings towards ACT as 
determined by responses to the Cockpit Management Attitudes Questionnaire 
(Helmreich Wilhelm, Gregorich, 1988). Responses to the following questions 


and statement were analyzed: 
¢ Overall, how useful did you find the training? 
¢ How important is recurrent training in aircrew coordination? 
¢ ACT has the potential to increase safety and crew effectiveness. 


¢ Will the training change your behavior on the flightdeck? 


Response options included: 
¢ A Waste of Time 
e Slightly Useful 
e Somewhat Useful 
e Very Useful 


e Extremely Useful 
90 percent of the participants found the course to be "very useful" or 
"extremely useful.” 80 percent believed recurrent training is "very useful" or 
“extremely useful." 86 percent agreed strongly with the statement that "ACT 
has the potential to increase safety and crew effectiveness." 67 percent 
believed a "moderate" or "large" change in their behavior would occur as a 
result of the course. The research also included an analysis of participant 


opinions of each module of the course. Overall, all modules were thought to 
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be useful by the majority of participants. Acceptance of the usefulness of P-3 
ACT was established for both officers and enlisted personnel. 

In their report Wilhelm and Helmreich provide a positive answer to the 
question; Will crewmembers have favorable feelings towards the concepts 
and theory that are the basis for ACT? However, proof that ACT is liked by 
those who attend is not proof that positive attitude changes have occurred. 
The next step in the analysis and the purpose of this thesis is to determine if 
ACT has changed attitudes towards flight safety. Utilizing a section of the 
CMAO separate from that used by Wilhelm and Helmreich, perceived 
attitude changes can be measured. Portions of the Cockpit Management 
Attitudes Questionnaire are designed to identify the attitude of the person 
taking the survey. P-3 ACT seminar participants were asked to complete the 
questionnaire before and immediately after the seminar. Changes in before 
and after questionnaire responses are indicative of an attitude change. If 
attitudes can be changed the course is serving its purpose of creating a more 


safe flying environment for the aircrews. 


H. ORGANIZATIONAL/INDIVIDUAL REACTION 
1. Resistance to Change 
Acceptance of the need for change through ACT requires agreement 
that past patterns of behavior were less than adequate. P-3 squadrons are 
heavily tasked with training and operational requirements and can only 
devote limited time to additional training. Their demanding schedule often 
requires extended working hours and work on weekends. Participation in 


ACT may require a crewmember with years of experience and thousands of 
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mishap free flight hours to add to an already busy work schedule. To achieve 
ACT goals, resistance to change due to time constraints and other reasons has 
to be overcome. 

The first step in overcoming resistance, is to identify a clear need for 
change. For ACT, the need for change is a need for preservation of lives and 
aircraft. This need may not be seen as a high priority for many P-3 
crewmembers. The length of P-3 service for the crewmembers surveyed for 
this thesis was less than 5 years. During their time of P-3 experience, few 
accidents have occurred. The excellent safety record of the P-3 relative to 
other Naval aircraft may promote a false sense of security. The ACT program 
must first establish with participants the need for change based on past 
mishaps that have occurred and the potential of this program to prevent 
further mishaps. 

The three-step process of crew coordination behavior change requires 
unfreezing of the present behavior pattern, moving to a new behavior 
pattern and refreezing desired behaviors (Lewin, 1958). The VP-31 course 
unfreezes present behavior through education of seminar participants. A 
review of P-3 and other crew coordination failures that have lead to mishaps, 
illustrates room for improvement. Movement to new behavior patterns is 
accomplished via knowledge and skills imparted through facilitators. 
Refreezing of desired behavior is accomplished through role play exercises 
where ACT skills are utilized by aircrew and then reviewed on video tape. 
The participation of senior crewmembers in the role play exercises is key to 
the acceptance of ACT concepts by junior personnel. Through their 


participation, senior personnel provide support for the legitimacy of the ACT 
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concepts. If the three-step change process has been successful new behaviors 
will be utilized during subsequent flights. 

To achieve the change, someone is required to serve as the change 
agent (Sziligy, Wallace, 1987). The change agent should be someone not 
bound by tradition, culture or the politics of an organization. This person 
must offer new ideas, viewpoints and perspectives. The change agent must 
have support from organizational leadership and be able to gain the trust of 
those for whom change is required. The change agent for P-3 ACT is Dr. 
Henry H. Smith of VP-31. As a civilian working for the Navy, Dr. Smith is 
outside the lines of authority affecting crewmembers. He has been involved 
in the ACT field of study since 1985. As a retired Navy Captain and an aviator 
with thirty plus years of experience, he is knowledgeable about flight 
operations and has sufficient status among aircrews to serve as a change agent 
for implementing ACT. 

Implementation of P-3 ACT has been accomplished with the 
awareness that resistance to change is expected. The primary means of 
Overcoming resistance is the validity and utility of the material to 
crewmembers. Change will occur if the material is useful. Also, selection of a 
credible change agent has provided the best opportunity for course concepts to 
be heard and understood. 

2. The Boomerang Effect 

Implementation of ACT into military aviation is not universally 
supported. For example, a P-3 flight engineer who has accumulated 
thousands of P-3 hours may not see a need to integrate new ideas into 


present methods of managing a cockpit. A P-3 ordanceman may feel that his 
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opportunity to affect safety of flight related scenarios is limited and his 
opinion is not necessary or desired by the pilot or flight officer. These and 
other aircrew who attend ACT seminars may have negative reactions to the 
material presented. 

A report on the Negative reaction phenomenon, When Training 
Boomerangs (Helmreich, Wilhelm, 1989), identifies a sub-group likely to 
have a negative reaction to ACT. The group is distinct not by the experience 
level or aircraft flown but by the personality type. Via factor analysis of the 
CMAQ, a group of individuals weak in both instrumental and expressive 
personality traits were identified. This group was prone to significant 
negative reaction to CMAQ communication and coordination related 
questions. ACT course concepts emphasize the importance and necessity of 
being a vocal and involved member of a crew. For this personality group 
these concepts conflict with established behavior patterns. 

The strength of negative reaction can be influenced by another factor, 
group dynamics. The effect of group dynamics influenced the degree a 
seminar was good or bad as seen by participants. A successful ACT seminar 
requires an engaged, participative crew focused on the material being 
presented. Many outside influences beyond those that the facilitators control 
can cause a seminar to fall short of this goal. When a seminar does not 
captivate the crew, the weak instrumental and expressive personalities are 
much more likely to reject material being presented. ACT can cause attitudes 
to shift opposite of the desired direction for a distinct group of individuals. 

P-3 ACT Facilitator training identifies the possibility of a negative 


reaction occurring. Facilitators are trained to identify aircrew who withdraw 
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from discussion or question the course content. Objections to the course 
concepts are encouraged to allow discussion among the group. Concepts are 
presented with "take it or leave it" option. Some will find ACT more useful 
than others. Even those who feel the course is of no use to them, have agreed 
that the course is useful to others (Smith, 1992). Boomerang individuals 
would be indicated in the CMAQ survey results by a shift in attitudes 
opposite from the desired direction. 

The boomerang effect and other effects of ACT can be measured by 
the CMAQ. How much change has occurred in attitudes of crewmembers 
can be determined through analysis of CMAQ data. The P-3 ACT data base is 


a resource for measuring attitudes of those who completed the seminar. 
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Ii]. DATA AND METHODOLOGY 


A. J DAT A Sid: 

A subset of the CMAO contains thirty-two questions designed to measure 
an individual's attitude towards crew coordination issues. For each of 1,200 
seminar participants, the CMAQ was administered just prior to and 
immediately after seminar completion. The sample of 1,200 represents 
approximately sixty percent of the total population who have completed the 
seminar. The data were screened based upon the criteria explained in part B 
of this chapter. The data were collected between June of 1988 and August of 
oe 

Survey results were tabulated by Dr. John Wilhelm, University of Texas 
at Austin, and transferred to the Naval Postgraduate School mainframe 
computer. The data set has been modified to allow completion of this 
research. Personality measures and narrative comments have been removed 
as they are beyond the scope of this report. The data set is now limited to 


seventy-one response variables for each seminar participant including: 
¢ Thirty-two preseminar attitude responses 
¢ Thirty-two postseminar attitude responses 


¢ Seven aircrew positional/experience demographic information 
variables 


B. FILTERING OF DATA 
Once the data file was loaded to the mainframe computer several actions 


were taken to ensure inaccurate data were filtered out. Data anomalies could 
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have occurred when surveys were completed, assembled into a data base or by 
Other causes. Data were screened to ensure the analysis could utilize 
information representative of P-3 aircrews. The following steps were taken 


to screen the data: 


e The CMAQ allows a response range of one through seven. Only those 
responses within this range were considered. 


¢ Observations with missing values in the survey responses were not 
considered in the analysis. 


¢ Number of hours with assigned crew was limited to 2000. 
¢ Number of P-3 flight hours was limited to 10,000. 
¢ Maximum age respondents was limited to a maximum of 45. 


e¢ Years in military service was limited to a maximum of 25. 

Limiting of responses by the above standards is intended to provide a 
population sample most representative of P-3 aircrews assigned to squadrons 
at the time the surveys were completed. Initially, 1,200 observations were 
screened to determine their utility for analysis. After application of the 
previously explained measures, the group was narrowed to 573 observations. 
The primary reason for exclusion in the analysis is failure to correctly fill out 
the survey. CMAQ responses out of the 1-7 allowable range and demographic 
data not consistent with reasonable crewmember profiles eliminated some 
from inclusion. Seminar facilitators labeled some seminars as particularly 
good and others as particularly bad. No effort was made to screen out 
seminars based upon the level of success measured by the facilitator’s 
evaluation. By including both good, bad and average seminars the analysis 


should reflect the overall program effects. 
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C RESPONDENT DEMOGRAPHIC INFORMATION 

The survey respondents represent the current pool of personnel the Navy 
has assigned operating P-3 aircraft in active duty squadrons. Variation in the 
demographic information is caused by organizational factors. All crew have 
differences in levels of experience based upon the years in the service and 
years of flying duty. Table 4.1 provides data on the respondents grouped by 
crew position. For each crew position provided are: hours assigned to crew 
(HRSCR), hours of P-3 flying (HRSP3), total flight hours (FLTHR), years of 
military service (YRSMIL), and age (AGE). 


TABLE 3.1 DEMOGRAPHIC DATA FOR PERSONNEL INCLUDED IN CMAQ 
ANALYSIS 


AVERAGE VALUES BY CREW POSITION 


POSITION bis Ck Se rksSrs PL ACs IR YRMIL AGE 
PLANE CMDR. es) 1544 2106 8.9 34 
SECOND PILOT 242 555 892 5.0 ai 
Peek) PILOT oy, 240 433 OHS) 28 
eel ENGK. SAG. 2416 25/9 TOS 29 
NAVIGATOR 162 630 829 5.3 ue) 
TACTICAL COORD. 188 1544 1700 7.8 58 
SEINSOK 1 290 1664 1766 8.1 28 
BeNOR 2 Pow 694 S88 4.3 26 
SENSOR 3 283 1562 eels 6.1 28 
PY PEIGHT TECH. Boo GZ 1160 6.6 30 
ORDINANCE 345 1091 SZ Dee 7, 
GROUP AVERAGE = 244 1360 1562 72 30 
Source: Data derived from the P-3 CMAQ survey. 


D. HYPOTHESIZED ATTITUDE CHANGE MODEL 

This report will determine if attitude changes have occurred as measured 
by the pre and post seminar CMAQ. Figure 3.1 provides a model of the 
process utilized to conduct this analysis. Initial seminar responses indicate 
preseminar attitudes towards concepts that make-up each CMAQ question. 


Initial predisposition is a result of an individual's knowledge, attitudes and 


ee: 


skill. Individuals are predisposed to certain attitudes as a result of experiences 
in the past. A difference in the responses indicates effects of ACT training on 
the respondent. The frequency and significance of changes in responses may 


reveal patterns associated with a particular CMAQ questions or a_ group of 


respondents. 
POST-SEMINAR PRE-SEMINAR CHANGE 
ATTITUDE - ATTITUDE = CAUSED 
RESPONSE RESPONSE BY ACT 


Figure 3.1 Hypothesized Measure of Attitude Change 


ACT seminar discussion is designed to create an increased awareness of 
crew coordination issues and promote a self-examination of existing attitude 
sets. Desirable and undesirable attitudes are explained by facilitators and 
demonstrated in role play activities. ACT is intended to reinforce desirable 
attitudes and eliminate undesirable tendencies. Participants who discover in 
themselves an undesirable attitude trait have a choice of reactions. The 
desirable reaction is for the individual to change his attitude to be more 
consistent with effective crew coordination. All phases of the seminar 
address attitudes either directly or indirectly through concepts taught. The 
seminar is designed to cause change in attitudes where change is necessary. 

Immediately following the seminar, before the participants depart, the 
CMAQ is re administered. CMAQ responses now become a measure of the 
attitude set an individual started the seminar with modified by changes 
caused by ACT. This report is focused on identifying changes that have taken 


place as a result of ACT. 
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E. ANALYTICAL PROCEDURES 
1. Hypothesis Tests For Means of Paired Samples 

An examination of changes in attitudes as measured by the CMAQ 
was conducted to determine changes caused by ACT. Thirty-two preseminar 
and postseminar CMAQ responses were compared for the group and for 
subgroups of respondents. Analyses included input only from those 
respondents not screened out by methods explained earlier in this report. T- 
tests applied to preseminar and postseminar responses were used to 
determine if changes in attitudes had taken place and if the changes were 
Statistically significant. T-tests by aircrew position were used to determine if 
ACT was more effective for particular subgroups. 

Preseminar and postseminar CMAQ scores are paired samples. By 
pairing the samples an accurate measurement of seminar effects can be 
obtained (Weiss, Hassett, 1991). The difference between the two responses is 


the paired difference and is defined as: 
ome cl = X_ xX] 


where, 


¢ x2 = postseminar and x1j= preseminar response 
The null hypothesis for the T-test analysis is that the difference 
between paired samples is zero. The alternative hypothesis is that the 
difference is greater than or less than zero. A significant change away from 
zero in the desired direction, indicates an attitude change has occurred. The 


null (Hg) and alternative (Ha) hypotheses are defined as: 
~ Ho: yd — p2 
weeeidias 1 > u2 or pl < p2 


oIS, 


By evaluating whether a change in response has occurred at the 5 
percent significance level, the standard by which an attitude change has 
occurred is determined. The t-tests applied were one tailed. The direction of 
the desired attitude shift is dependent upon the CMAQ question. Questions 
for which a positive shift is desired were right tailed. Questions for which a 
negative shift is desired were left tailed. Both tests were done at the 5 percent 
significance level as signified by: 

et =_05 

For this analysis, the significance level of 5 percent was used to 
determine if the null hypothesis was accepted or rejected. Testing at the 5 
percent level 1s a commonly used value although more or less stringent 
criteria can be applied. When all participants were analyzed together the 
critical value of t is 1.645. For subgroups of twenty-nine or less the critical 
value increases. In this thesis group size exceeded twenty-nine for all groups 
allowing the same critical value to be used for all t-test analyses. The critical 


value is: 
© 19.95 = 1.645 if n> 29 


A hypothesis test can be completed utilizing the mean of the 
paired differences, the standard deviation of the paired differences and a sum 
of observations utilized. The result of the relationship of these variables is a 
test statistic that can be compared against the critical value. If the test statistic 
does not exceed the critical value the null hypothesis is accepted. If the test 
statistic exceeds the critical value the null hypothesis is rejected. The test 


Statistic is determined by: 


d 
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2. Correlation Analysis 

A correlation analysis of changes in survey responses and 
demographic data was used to indicate the relationship between these 
variables and the responses given. Years of flying and military experience 
may correlate with an increased or decreased acceptance ACT concepts. 
Correlation analysis can detect any linkage between CMAQ responses and 
flight hours, P-3 experience, age and years of military service. The Pearson 
product moment correlation coefficient is used to provide the level of 
correlation between CMAQ responses and individual characteristics (Weiss, 
Hassett, 1991). The value of the correlation coefficient (r) will lie between -1 
and 1. Values close to -1 or 1 indicate strong correlation. Values close to zero 


indicate weak correlation. Correlations were computed via the following 


formula: 
Sx 
ee ae 
e Yr = 
SxxSyy 
where 


© Syx = Lx (Lx)? / 1 
© Sxy = Lxy- (Xx) Ly) /n 
© Sy = Ly? - (Ly)? / n 
Correlation analysis can identify strengths or weaknesses in ACT 
training. If certain attributes of entering experience or qualification level 
affect ACT outcomes, those attributes will result in exceptional correlation 
scores. Correlation analysis should indicate the effectiveness of training for 


different sub-groups of participants. 
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IV. RESULTS AND DISCUSSION 


This section describes the results of the analyses undertaken to determine 
if attitude changes occurred as a result of ACT for P-3 aircrews. The primary 
research objective is to apply statistical analysis techniques to the CMAQ data 
base and interpret results. Results for group and sub-groups therein will be 


determined. 


A. T-TEST ANALYSIS 

Before and after seminar survey responses are the focus of this thesis. 
Table 4.1 presents analysis results of preseminar and postseminar responses 
for the entire group of respondents. Results for sub-groups by aircrew 
position are in appendix B (1-11). Differences between responses before and 
after the seminar are indicative of attitude changes. For half of the CMAQ 
responses (Questions 2,3,4,6,7,8,10,12,14,15,16,20,21,22,23,28) a positive value 
shift in survey response is desirable. For other survey responses (Questions 
1,5,9,11,13,17,18,19,24,25,26,27,29, 30,31,32) a negative value shift is the desired 
outcome. The difference in the desired direction is caused by survey question 
wording. Some questions evaluate concurrence with desirable attitudes, 
others evaluate level of disagreement with undesirable tendencies. This 
analysis will focus on changes in survey responses to evaluate their frequency 
and significance. The analysis is conducted for the entire group and for sub- 
groups. 

T-tests of the difference between the thirty-two preseminar and 


postseminar CMAQ responses indicate attitude changes caused by ACT. 
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Where the test statistics did not exceed the critical value no change in attitude 
was judged to have occurred. When the test statistics exceeded the critical 
value an attitude change was judged to have occurred. For each individual 
thirty-two possible changes could have occurred on the CMAQ. Based on the 
5 percent significance level standard the total number of changes occurring 


was determined. Three possible outcomes existed for each CMAQ response: 
e An attitude shift in the desired direction 
e No attitude change 


e An attitude change opposite the desired direction 
Table 4.1 provides the combined t-test results for all crew positions 
grouped together. The combined t-test is useful for determining the impact of 
ACT on the group as a whole. The methodology used to attain the results in 
table 4.1 is explained in chapter III section E of this report. The last column in 
table 4.1 indicates the probability value and whether or not the value was 


significant. 


og 


TABLE 4.1 GROUP CMAQ PRESEMINAR (x1) AND POSTSEMINAR (X2) 
SURVEY MEANS, CHANGE, T VALUE AND T-TEST RESULTS 


CMAQ X1 X2 IDEAL ACTUAL T PROBABILITY 
MEAN MEAN} SHIFT GHANGE ©VAEUE. VRBE 
1 4.40 3.68 - -.717 11.943 0001 * 
Z 5.67 D5 + .283 5.500 0001 * 
5 Soy) 3.96 + 394 7.940 0001 * 
4 4.36 4.30 + -.067 1.001 3171 
5 5.34 4.46 - -.869 10.642 0001 * 
6 6.02 6:27 + 248 6.013 O00." 
7 6.73 6.76 + UBS: 1.369 MAE 
8 S58: 6.07 + 476 10.417 0001 * 
9 62 30 - -.519 9.634 0001 * 
10 6.32 6.46 + 136 3.660 .0003 * 
i 3228 2.42 - =855 10.625 .0001 * 
12 a) Deb + .216 4.987 0001 * 
js 4.32 4.4] - 089 1.428 0001 * 
14 aU 6.03 ti 1.016 14.717 OOO] * 
15 2.85 ZU + 065 0.848 3966 
16 Do 6.38 + 1395 8.290 0001 * 
WE 4.11 S88: - -.276 4.568 0001 * 
18 2s 226 - .080 1392 1645 
12 2.40 DDS - -.148 2.364 .0184 * 
20 5.41 6.04 + .640 el 0001 * 
21 Gall 6.66 + .150 4.437 0001 * 
22 Sis, 6.33 + 77, 737, 0001 * 
23 D.o2 balls + 317 Se dole, .0001 * 
24 4.15 3.61 - -.543 7.949 0001 * 
25 5.05 4.87 - -.185 3.167 .0016 * 
26 4.78 4.57 - -.200 2.752 0061 * 
oT, SNe FE S7 - -.035 0.670 9031 
28 Gazz 6.48 + .260 5.394 .0001 * 
29 2.08 eZ - -.361 7 NOM ‘OOO 
30 2.48 2.24 - -.240 4.532 0001 * 
31 2.02 Lae. - ao, 5.708 .0001 * 
By" 1.22 1.24 - .016 0.651 5154 
Source: Data derived from the P-3 CMAQ surveys. 


* Indicates desirable attitude change occurred. 
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The combined group t-test indicated 25 desirable CMAQ changes out of 32 
possible. Utilizing the entire group for the test, provides a much higher 
number of indicated attitude changes as compared to t-tests by sub-group. The 
higher number is caused by the combined effect of the group. A slight though 
insignificant change in attitude for a sub-group may become a significant 
change when all the groups share the same slight change in attitude. 25 out 
of 32 desirable shifts in attitude represent a change on 78 percent of the CMAQ 
questions. 

Table 4.2 provides the T-test results for the entire survey group as 
indicated by table 4.1 and for sub-groups by aircrew position. Table 4.2 
indicates the number of significant attitude changes occurring out of a total of 
thirty-two possible. Significant changes in attitudes for sub-groups were 
determined by the same method used in table 4.1 based on the results in 


aowendix B 1-11. 
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TABLE 4.2 BREAKDOWN OF ATTITUDE CHANGES AS INDICATED BY A 
T-TEST OF PRE AND POST ACT SEMINAR CMAQ SURVEYS 


CMAQ RESPONSES 


DESIRABLE NO UNDESIRABLE 
GROUPING Sai CHANGE SHIFTS 
ALL RESPONDENTS 25 yi 0 
PLANE COMMANDER 16 NS) 1 
SECOND PIO 7 29 0 
THIRD PILOT 7 25 0 
FLIGHT ENGINEER 16 16 0 
NAVIGATOR 13 i 0 
TACTICAL COORDINATOR _ 18 14 0 
SENSOR ONE 10 22 0 
SENSOR TWO 10 pa. 0 
SENSOR THREE 11 J) 0 
INFLIGHT TECHNICIAN 14 18 0 
ORDNANCE 1] ZA 0 


Source: Derived from analysis of P-3 CMAQ data. 


The results of t-test analyses as displayed in table 4.2 indicate that in many 
cases the null hypothesis that CMAQ scores are the same after the seminar, 
can be rejected. The range of significant desirable attitude shifts for all 
positions is 7 to 18 for the 32 CMAQ questions. A comparison of attitude 


changes is displayed in figure 4.2. 
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GROUP SCORE 
TACTICAL COORDINATOR 
PLANE COMMANDER 
FLIGHT ENGINEER 
IN-FLIGHT TECHNICIAN 
NAVIGATOR 

SENSOR THREE 
ORDNANCE 

SENSOR ONE 

SENSOR TWO 

SECOND PILOT 

THIRD PILOT 


oe 
Le 


CMAQ CHANGES 


Gal 
Ea 
C] 
CJ 


P-3 AIRCREW POSITIONS 





Figure 4.1 Favorable Changes in CMAQ Attitudes 


B. INTERPRETATION OF T-TEST ANALYSIS 

The CMAQ is an instrument for measuring concurrence with attitudes 
conducive to effective crew coordination. The method used to construct the 
CMAQ has provided a tool for measuring overall crew coordination 
awareness and potential. It was not designed to directly identify particular 
strengths or weaknesses in sections of P-3 ACT. The material in P-3 ACT does 
not directly address questions on the CMAQ. Within each CMAQ question 
are many crew coordination concepts and issues. Question by question 
analysis of CMAQ results does not allow a distinct understanding of thirty- 
two separate ACT concepts and issues. By looking at the combined CMAQ 
change for all thirty-two questions the effect of ACT can be seen. The results 
of the P-3 CMAQ have provided the change in CMAQ scores based on 


comprehensive change and not a question by question analysis. P-3 CMAQ t- 
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test results have resulted in the formation of three distinct groups. The three 
sub-groups are identified and explained in the following sections. 
1. Sub-group One 

Plane Commanders (16), Flight Engineers (16) and Tactical 
Coordinators (18) had the highest number of attitude shifts resulting from 
ACT training. A Plane Commander is primarily responsible for the safe 
completion of a flight. The Tactical Coordinator is responsible for successful 
completion of a mission. The flight engineer is responsible to the pilot for 
support in flight safety related issues. Successful completion of all three of 
these crew assignments is dependent upon effective crew coordination. 
These results indicate a positive relationship between responsibility for crew 
coordination and the impact of ACT. 

2. Sub-group Two 

Figure 4.1 indicates a similar level of attitude changes across sensor 
station operators, inflight technician and ordnanceman. For this group the 
average number of attitude response changes was 11.2 indicating that for over 
one third of the CMAQ responses an improvement on Safety attitudes was 
achieved. During the conduct of a flight these crew positions can occasionally 
play a critical safety role during a flight. However, for most flights they are 
more likely to play minor role related to safety issues. To this group, ACT 
concepts are more likely to reflect a new way of thinking about their roles 
because their training places less emphasis on crew coordination and safety. 
This subgroup of the P-3 crew is mainly responsible for mission related 
aspects. The P-3 ACT course is designed to broaden utility of ACT from flight 


station personnel to include the all crewmembers. The number of positive 


shifts occurring in this group supports the ACT principle that crew 
coordination is everyone's responsibility. 
3. Sub-group Three 

second Pilots, Third Pilots and Navigators represent another 
subgroup of attitude shifts as measured by the CMAQ. A commonalty within 
this group is the accomplishment of tasks under the immediate direction of 
others on board the aircraft. The Second and Third Pilots perform their 
assignments under the immediate direction of the Plane Commander. The > 
Navigator works under the supervision of the Tactical Coordinator. For all 
three crew positions, tasks, are to be carried out according to guidelines set by 
the Plane Commander or Tactical Coordinator. Crew coordination 
development and execution is a lessor role for this group. The lower number 
of shifts for Second Pilots, Third Pilots and Navigators as compared to their 
senior counterparts, could be reflective of a lessor crew coordination 


responsibility and authority. 


C CORRELATION ANALYSIS 

Identification of subgroups for whom ACT is more or less effective, 
would be useful for understanding the effects of ACT. Perhaps ACT is not 
effective for sensor station operators who are less involved in safety issues. 
Possibly senior personnel will resist the new concepts and revised thinking 
that is part of ACT. Previous safety related programs focus on flight station 
personnel only whereas ACT has brought in all crew positions. Correlations 
were run on demographic variables and the difference scores between pre- 


and post- CMAQ responses. These analyses will help support or dispel these 
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and other questions regarding ACT and its relationship to demographic 
information included in table 3.1. 

Correlation responses fall within a -1.00 to 1.00 range. Any value near 
these extremes can be evaluated for its implications for seminar design and 
teaching. Table 4.3 provides a correlation analysis for change in CMAQ score 
and four demographic variables including: hours assigned to crew (HRSCR), 
hours of P-3 flying (HRSP3), years of military service (YRMIL) and birth year 
(AGE) for the all respondents. 
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TABLE 4.3 CORRELATION OF CMAQ CHANGES AND DEMOGRAPHIC 
DATA CORRELATION VALUES 


CMAQ CHANGE HRSCR HRSP3 YRMIL AGE 
1 08 08 —.02 —.04 
Ya 06 —.04 —.04 —.08 
5 = (02 06 a2 O1 
- 00 00 00 00 
3 = 1) 04 07 06 
6 —.06 —.03 -.03 00 
7 00 00 —.04 00 
8 = =e —.10 =..\) 
S -—,|() =|) —.04 = 05 

10 -.02 -.01 —.02 Sb 
11 -.10 —.06 04 07 
12 -.01 —.04 —.04 00 
13 02 00 03 —.02 
14 08 —.06 = US —.02 
ils: 03 —.02 03 00 
16 05 .08 07 04 
17 05 .O1 05 03 
18 —.01 04 .O3 00 
iy, = 16 03 Me 05 
20 03 —.02 .OO -.01 
74 10 107 a7 00 
a2 06 {yy {uk -.01 
28) —.04 .O7 08 08 
24 03 OM 06 03 
BS OS 03 A09 06 
26 -.O1 04 00 -.02 
27 03 08 .Q2 00 
28 —.02 —.06 = 07 06 
US = 108 05 nt, 02 
30 00 = [Ue = 04 S08, 
31 —.07 —.03 05 05 
OZ O1 —.01 OO 02 


source: Derived from analysis of P-3 CMAQ data. 


47 


No significant correlation exists between CMAQ response changes and 
the demographic variables as displayed in table 4.3. The highest correlation 
value in table 4.3 is .10. Even the highest value does not identify any strong 
correlation between attitude change and the demographic variables included 


in the analysis. 


D. INTERPRETATION OF CORRELATION ANALYSIS 

The CMAQ was not specifically designed to allow correlation of 
individual background data and changes in attitudes. Inability of the CMAQ 
to indicate correlation of individual data and changes in CMAQ response 
does not mean that another survey instrument could not demonstrate this 
relationship. Correlation of hours assigned to crew, P-3 flight hours, years in 
the military and age with the relatively small average change in CMAQ 
responses results in low correlation values. For all but one of the thirty-two 
CMAQ questions the average change in survey response was less than 1.0 for 
the group analyzed (table 4.1). The small range of average change in survey 
response offers minimum opportunity for establishing a correlation with 
CMAQ score and the demographic variables used in the analysis. Based upon 
the change in CMAQ response that did occur, no significant correlation 
existed between change in response and individual characteristics. 

Correlation analysis indicates CMAQ response is not a function of P-3 
flight experience, hours assigned to a crew, years military, and age. Lack of 
strong correlation (table 4.3) indicates these variables do not significantly 
affect responses to the CMAQ. Attitude changes could not be predicted by an 


individual's experience level. Based on the correlation analysis, aircrew of 
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different experience levels react similarly to ACT. No experience level group 
is reacting significantly better or worse to the seminar. No change in the way 
ACT is designed or instructed is needed based upon years in the military, age, 


hours on a crew or P-3 flight hours. 
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VI. CONCLUSIONS AND RECOMMENDATIONS 


A. CONCLUSIONS 
1. CMAQ T-test Results 

P-3 ACT was successful in improving the attitudes of crewmembers 
who attended the seminar. The present format of the seminar is creating an 
understanding of attitudes necessary for effective crew coordination. Based 
on knowledge attained in the seminar, crewmembers are changing their 
CMAQ responses. The t-test results do provide evidence that attitudes are 
changing. P-3 CMAQ results indicate that movement away from attitudes 
that have caused mishaps in the past. Further, movement on CMAQ 
responses is substantial and is occurring for all crew positions. Additional 
analyses using the same methodology could provide a basis for evaluating the 
P-3 seminar relative to other programs. 

Some differences existed by crew position in the number of changes 
that occurred on the CMAQ (table 4.2). Crewmembers in leadership roles 
were most likely change their attitudes as a result of ACT. Those less 
involved in managing crew coordination activity were less likely to indicate 
an attitude change. The value of ACT seems to be most appreciated by those 
who are normally responsible for implementing ACT concepts. Changes 
away from desirable crew coordination attitudes are not evident in the CMAQ 


analysis. 
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2. CMAO Correlation Results 

The existing course content and seminar format are uniformly 
effective across aircrew experience levels. The correlation analysis (table 5.2) 
fails to identify any anomalies associated with CMAQ response and the 
crewmember demographic variables. The seminar does not teach to a 
particular experience level group at the expense of others. Hours of P-3 flying, 
hours assigned to crew, years in the military and age were not useful in 
predicting changes in attitudes. The correlation analysis supports the concept — 
of bringing in an intact combat aircrew for the seminar. 

By attending ACT as a combat crew, a group attitude may be formed 
towards crew coordination. The group attitude towards ACT may result from 
the combined effects of individual attitudes. The group coordination attitude 
is important as the group will encounter challenges together. Knowledge of 
an individual member's attitude towards crew coordination can be used to 
make the group more effective under challenging flight scenarios. This 
analysis supports the present scheduling by complete combat aircrew as 
opposed to by aircrew position. 

3. Program Effectiveness 

P-3 ACT has substantive indications of being an effective program as 
measured by outcome, process and moderator criteria. Indicators include: 
CMAQ attitude changes, avoidance of a mid-air collision (Smith, 1992) and 
continued high-level organizational support.  P-3 flight manuals now 
include ACT as a critical sub-area for success on the annual flight evaluation. 
To be designated as a qualified crewmember, crew coordination proficiency 


must be demonstrated. Integration of ACT into flight evaluations is an 


Dil 


indication of commitment to the program. The substantial dedication of 
resources over several years has allowed program development and 
implementation. Continued implementation of ACT will eventually allow 
the necessary time to evaluate ACT effects on P-3 mishap rates. In time the 


benefit of ACT could be put in terms of number of lives and aircraft saved. 


B. IMPLICATIONS 

The Naval Safety Center has identified crew coordination error as the 
number one causal factor in Naval Aviation mishaps. An increase in 
effective crew coordination capability creates the potential for reduction of 
crew coordination related mishaps. The P-3 ACT course is capable of 
improving crew coordination attitudes as measured by the CMAQ. 
Eventually a reduction in P-3 crew coordination related mishaps should occur 
if attitude changes seen here transfer to behavior changes on the job. An 
accurate measure of this reduction can only be made after sufficient time has 
taken place to account for the chance element that is present in tracking the 


occurrence of mishaps. 


C. RECOMMENDATIONS 

The initial analysis of the P-3 program completed in 1990 by Wilhelm and 
Helmreich (Wilhelm, Helmreich, 1990), determined that crewmembers 
believed P-3 ACT training was useful. This thesis indicates that in addition 
to finding the course useful, crewmembers attitudes are changed in a positive 
direction by the course. A follow-on thesis could attempt to determine if crew 


coordination concepts are being utilized. 
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Use of ACT could be determined by surveying seminar graduates and 
requesting them to document situations where ACT was used. The feedback 
from aircrew operators would be valuable in documenting the application of 
ACT during actual flight operations. Documentation of actual events could 
allow the determination of the effectiveness of separate ACT modules. The 
follow-on thesis or report could be constructed of case studies that emphasize 
the use of ACT during flight operations. 

The CMAQ used to analyze the P-3 crewmember responses could be 
redesigned or expanded to take into account factors unique to Naval 
Aviation. The CMAQ was designed based on the analysis of civilian aviation 
incidents. Perhaps a similar study of Naval Aviation mishaps would reveal 
crew coordination concepts unique to Naval Aviation. A redesigned CMAQ 
based on crew coordination issues most relevant to Naval Aviation could 
provide a more useful measure. The present CMAQ is valuable in providing 
a standardized measure for the effectiveness of crew coordination training. A 
revised CMAQ modified for Naval Aviation could be used as a_ standard for 
crew coordination training. 

The CMAQ should be revised so that all questions evaluate concurrence 
with desirable attitudes. At present the CMAQ measures concurrence with 
desirable crew Coordination attitudes (Questions 
2,3,4,6,7,8,10,12,14,15,16,20,21,22,23,28) and disagreement with undesirable 
attitudes (Questions 1,5,9,11,13,17,18,19,24,25,26,27,29, 30,31,32). Questions 
could be restructured so that for all questions a desirable shift in attitude 


occurs in the same direction on the survey scale. Analysis and presentation 


Do 


of survey data would be more clear if all questions were consistent in 
evaluating concurrence with desirable attitudes. 

The completion of ACT training brings the program to a transition point. 
The one-day seminar format for P-3 ACT has been presented to all Moffett 
Field California and Barbers Point Hawaii P-3 squadrons. The decision to 
continue or discontinue training must be made. If training is continued the 
new content and format must be determined. Reduced benefit may be 
realized by attending the same course a second time. A revised version of the 
course may need to be developed. A longitudinal study could determine the 
durability of the effects of ACT and the present need for continued training. 

Options for continued ACT training include the use of simulator and 
flight training. Limited flight hours and simulator time make it unlikely that 
entire events could be exclusively dedicated to ACT training. A practical way 
of including ACT concepts as an integral part of flight and simulator events is 
via the evaluators who conduct these events. All P-3 flight crew personnel 
are required to pass an annual flight evaluation. The evaluators who conduct 
these events could provide the means for further ACT training. For this to 
occur, evaluators must first complete a follow-on ACT course that imparts to 
them a deeper understanding of ACT principles along with the ability to 
evaluate the crew coordination skills of others. The evaluators could then 
instruct and evaluate crew coordination as part of every flight or simulator 
eVenir 

Aircrew Coordination Training is sponsored within the Navy for the 
purposes of achieving a reduced mishap rate. Avoidance of mishaps in part 


depends upon effective crew coordination. Effective tactical utilization of the 
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P-3 aircraft also demands effective crew coordination. The ability of P-3 ACT 
to enhance the tactical ability of a combat aircrew has not been determined. If 
ACT equally applies to tactical crew coordination the scope of the course may 


be expanded to enhance safety and tactical ability. 
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APPENDIX A. P-3 COCKPIT MANAGEMENT ATTITUDES 


UT/NASA/Navy 5-89 QUESTIONNAIRE (CMAQ) Page 1 
P-3 AIRCREW SURVEY 


I. Patrol Crewmember Attitudes 


As part of NASA sponsored research, we are collecting data on current 
attitudes in military operations. This survey measures your thoughts and 
eee. it is not an assessment of your learning. All data will remain 
strictly confidential! The identification number allows us to relate your 
future surveys to this one. 


Please answer by writing beside each item the number from the scale 
below that best reflects you 6 Note: when we use the words 


a or “crew members", we mean everyone on the aircraft unless otherwise 
noted. z 
KKKEKEEKS cal a kkk 
1 2 3 4 5 6 7 
2 9 Heer cc eee nne eee ee cee tecereneenne +---------- + 
Disagree Disagree Disagree Neutral Agree Agree Agree 
Completely Mostly Somewhat Somewhat Mostly Completely 


Strict utilization of the chain of command is essential for effective 
crew performance. 


Aircrew members should feel obligated to mention their own 
pe yeueveel cer stress or physical problems to other aircrew members 
efore or during a aission. 


It is important for all crew members to provide constructive 
criticism about the procedures and eeaneiques of others. 


I am more prone to make minor mistakes during periods of high 
workload than I am in routine mission situations. 


The PPC is primarily responsible for the safety of each mission. 


Each crew member should monitor other crew members for tft of 
SR ECES or fatigue, and should discuss the situation with the crew 
member. 


Good communications and crew coordination are as important as 
technical proficiency for the safety of flight. 


Aircrew members should be aware of and sensitive to the personal 
problems of other crew members. 


The PPC should take control and fly the aircraft in emergency and 
non-standard situations. 


The pilot flying the aircraft should pe plans for procedures 
or maneuvers and should be sure that the ormation is understood 
and acknowledged by crew members affected. 


Copilots and other crew members should not question the decisions or 
pea ot the PPC except when these actions threaten the safety of 
the ght. 


Crew members should alert others to their actual or potential work 
overloads. 


Even when fatigued, I perform effectively during critical flight 
maneuvers. 


PPC’s should encourage copilots, flight engineers, and other crew 
members to question procedures during normal flight operations and in 
emergencies. 


There are no circumstances (except total incapacitation) where the 
copilot should assume command of the aircraft. 


A debriefing and critique of procedures and decisions after each 


mission is an important part of developing and maintaining effective 
aircrew coordination. 
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UT/NASA/Navy 5-89 ace? 


KKKKEKKS Cal ekeekee 


1 2 3 4 5 6 7 

wees eeeene DOS GS yo oS SO te Bg 26 Soo Se Cn ee ee eae Pe ee aes 
Disagree Disagree Disagree Neutral Agree Agree Agree 
Completely Mostly Somewhat Somewhat Mostly Completely 


My performance is not adversely affected by working with an 
inexperienced or less capable crew member. 


Overall, successful mission accomplishment is primarily a function of 
the aircraft commander's flying proficiency. 


Correcting the procedures and techniques of others should be avoided 
since it can lead to tensions between crew members. 


Crew members should voice their concerns even if they are contrary to 
decisions which have alredy been made. 


The pre-mission aircrew briefing is important for safety and for 
effective crew management. 


Effective crew coordination requires crew members to take into 
account the personalities of other crew members. 


All crew members should share responsibility for prioritizing 
activities in high workload situations. 


A truly Pe ckess Ones aircrew member can leave personal problems 
behind when flying a mission. 


BY decision making ability is as good in emergencies as in routine 
mission situations. 


Training seldom interferes with safe and effective mission 
accomplishment. 


peadenea’P of the aircrew team is expected to come solely from the 
Enlisted crewmembers’ questions and suggestions should be considered 
by the flight deck. 


When joining a crew for the first time, a new crew member should not 
offer suggestions or opinions unless asked. 


It is better to let someone do their job the way they are used to 
rather than offering what you believe to be a better solution. 


Because flight engineers have no pilot training, they should limit 
their attention to airplane systems. 


PPCs who accept and implement suggestions from the crew are lessening 
their stature and reducing their authority. 


on 


APPENDIX B-1. T-TEST RESULTS FOR CMAQ RESPONSES 


TABLE B-1. PLANE COMMANDER T-TEST RESULTS 


a Nd Il POSIT=1] ------ oe oe ne ee er: 
Variable N Mean Std Error T Prob>I|TI 
QDIFF1 96 —On 515 0.138 -5.906 0.0001 
QDIFF2 96 0.292 0.099 2.939 0.004] 
QDIFF3 96 O.271 0.120 2.260 0.0261 
QDIFFS 96 -0.083 0.142 -0.588 0.5581 
QDIFF5 95 -1.197 0.183 -6.261 0.0001 
QDIFF6 96 0.219 0.093 2.359 0.0204 
QDIFF7 96 0.042 0.051 0.815 0.4171 
QDIFF8 96 0.354 0.104 3.417 0.0009 
QDIFF9 96 =—O57 5 0.137 -4.178 0.000] 
QDIFF10 96 0.125 0.080 1.56] 0.1219 
QDIFF1] 96 i575 0.163 roi UG 0.0007 
QDIFF12 96 0.167 0.086 1.940 0.0554 
QDIFF13 96 -0.073 0.155 -0.497] 0.6387 
QDIFF1S 95 0.863 0.133 6.505 0.000] 
QDIFF15 95 0.105 0.167 0.630 0.5299 
QDIFF16 95 0.400 0.123 3.256 0.0016 
QDIFF17 96 -0.385 0.160 -2.413 0.0178 
QDIFF18 96 -0.010 0.109 —0,095 0.9293 
QDIFF19 96 -0.063 0.109 =0 .575 0.5664 
QDIFFZ0 96 0.708 0.125 5.680 0.0001 
QDIFF21 96 0.115 0.089 hecge 0.1995 
QDIFF22 96 0.156 0.110 1.419 0.1593 
QDIFF23 96 0.4958 0.126 3.625 0.0005 
QDIFF24 96 -0.396 0.151 aie 15) es) 0.0100 
QDIFF25 96 =0'. 02] 0.132 -~0.158 0.8746 
QDIFF26 95 =, 0535 pe leAsid =0-505 0.7614 
QDIFF27 96 =). 052 ~ 0.103 -0.506 0.6142 
QDIFF28 96 0.083 0.090 0.929 0.3551 
QDIFF29 96 -0.28] 0.135 -2.088 0.03994 
QDIFF30 96 ieee 0.114 -].097 0.2756 
QDIFF31 96 -0.094 0.099 -0.943 0.348] 
QDIFF32 96 0.115 0.059 1.942 0.0551 
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TABLE B-2. SECOND PILOT T-TEST RESULTS 


ee = Sa Se ee eS Blt. ee 
Variable N Mean Std Error T Prob>{T| 
QDIFFl 23 -0.870 0.363 me OSA 070255 
QDIFF2 Zo 0.174 0.279 0.624 0.5390 
QDIFF3 25 0.522 0.266 1.963 0.0624 
QDIFFS4 23 02522 0.258 2.021 040557 
QDIFF5 ZS —( 4955 0.506 -0.859 0.3996 
QDIFF6 23 0.304 0.171 ey {7 pas) 0.0897 
QDIFF/ 23 0.000 0.089 0.000 1.0000 
QDIFF8 23 0.348 0.195 17365 0.088] 
QDIFF9 23 =(i5 565 0.287 -} .970 0.0616 
QDIFF1O 22 -0.136 0.119 1.142 0.2664 
QDIFF11 23 0.087 0.503 0.173 0.8642 
QDIFFl2 23 0.478 0.226 2.21 0.04955 
QDIFF13 Z5 0.304 0.347 0.877 0.3897 
QDIFF14 22 Ls 2 0.346 3.4917 0.0026 
QDIFF15 cc 0.045 0.392 0.116 0.9088 
QDIFF16 22 0.136 0.136 1.000 0.3287 
QDIFF17 23 -0.4978 0.3499 -1.369 0.1848 
QDIFF18 23 0.261 0.19] L367 0.1855 
QDIFF19 23 0.130 0.334 0.390 0.7003 
QDIFF20 22 OV727 0.349 2.062 0.0497 
QDIFF21l 23 0.130 0.114 1.141 0.2660 
QDIFF22 23 0.130 0.170 0.768 0.4509 
QDIFF23 23 0.391 0.2249 1.744 0.0951 
QDIFF2S4 23 =03705 0.259 -3.023 0.0063 
QDIFF25 23 -0.348 0.324 =O 7S 0.2951 
QDIFF26 22 -0. 72/7 0.256 =2.835/ 0.0099 
QDIFF27 25 -0.304 0.284 -1.071 0.2958 
QDIFF28 Z35 0.000 0.126 0.000 1.0000 
QDIFF29 23 -0.304 02255 -1.194 0.2451 
QDIFF30 25 -0.130 0.211 -0.617 0.5435 
QDIFF31 23 -0.043 0.172 -0.253 0.8027 
QDIFF32 23 0.043 0.076 0.569 0.5753 


wae ew wee es SO OS ST SB SBS SB SS VSO SOO SOS SF SF SF SF SF SF SS SST VFO BS VB SOOT FB 82 S| SSS SS SST 8 SF SSO SS — 


oy 
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TABLE B-3. THIRD PILOT T-TEST RESULTS 


“<<< <=95-"=5>5——>—— — = — aie ea POSIT=3 corr rrr rrr rrr errr tc r ccc ccccc- 
Variable N Mean Std Error T Prob>|T| 
QDIFFI] 34 -].000 0.223 -G .G476 0.000] 
QDIFF2 34 0.176 0.107 1.643 0.1098 
QDIFF3 34 0.494] 0.175 2.520 0.0167 
QDIFFS 34 -0.176 0.225 —(0.. 753 0.4390 
QDIFFS5 34 =0-.971 0.314 -3.086 0.0049] 
QDIFF6 34 0.265 0.195 1.358 0.1836 
QDIFF7 34 -0.118 0.092 =].2/7/7 0.2107 
QDIFF8 33 0.182 0.171 1.063 0.2959 
QDIFF9 34 =(noo7 0.188 -G.701 0.000] 
QDIFF10 34 -0.029 0.16] —-0.183 0.8559 
QDIFFl1 34 =0347:5> 0.314 ~2.342 0.0254 
QDIFFl2 34 0.206 0.157 1.314 0.1980 
QDIFF13 34 -0.029 0.237 -0.124 0.9019 
QDIFF14 32 0.906 0.285 52.177 0.0034 
QDIFF15 32 0.219 0.268 0.815 0.49213 
QDIFF16 32 0.28] 0.197 1.4928 0.1632 
QDIFF17 34 -0.500 0.265 -].890 0.0675 
QDIFF18 34 -0.059 0.286 -0.206 0.8384 
QDIFF19 34 0.176 O722) 0.797 0.4309 
QDIFF20 34 0.118 0.270 0.4936 0.6654 
QDIFF2] 34 -0.029 0.099 -0.297 0.7680 
QDIFF22 34 0.265 0.154 1.719 0.095] 
QDIFF23 34 0.412 0.257 1.601 0.1189 
QDIFF24 34 -0.497] 0.268 eed 7 0.0882 
QDIFF25 34 -0.353 0.289 ~)occl 0.2308 
QDIFF26 33 -0.364 O:225 -].614 0.1165 
QDIFF27 34 0.059 O25 0.274 0.786] 
QDIFF28 33 0.303 0.119 2.545 0.0159 
QDIFF29 34 =-072.353 0.179 =e 977 0.0565 
QDIFF30 34 0.029 0.1493 0.206 0.8384 
QDIFF31 34 -0.206 0.125 -].646 0.1093 
QDIFF32 34 -0.029 0.067 -0.4942 0.6615 
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TABLE B-4. FLIGHT ENGINEER T-TEST RESULTS 


= = == === = SS = POSIT=4Q -n-e mmm mn mn nn nr ree 
Variable N Mean Std Error T Prob>IT| 
QDIFFl 63 -0.44G 0.182 -2.4G0 0.0175 
QDIFF2 63 0.048 0.162 0.293 0.7702 
QDIFF3 63 0.349 0.150 Zeoce 0.0235 
QDIFFS 63 -0.254 0.235 -1.080 0.2843 
QDIFF5 63 -0.762 0.269 -2.828 0.0063 
QDIFF6 63 0.032 0.104 0.306 0.7603 
QDIFF7 63 0.016 0.089 0.178 0.8591 
QDIFF8 63 023517 0.126 Leoel 0.0141 
QDIFFI 63 -0.603 0.144 -G4.187 0.0001 
QDIFF10 63 =Q 2127 0.110 -~1.158 0.2514 
QDIFF1l1 63 -0.492 022235 -2.209 0.0309 
QDIFF12 63 0.238 0.127 1.868 0.0664 
QDIFF13 63 0.460 0.184 2.507 0.0148 
QDIFF14 61 0.672 0.237 2.837 0.0062 
QDIFF15 61 0.213 0.197 1.081 0.2840 
QDIFF16 61 0.426 0.116 3.687 0.0005 
QDIFF1?7 63 -0.032 0.207 =0,153 0.8788 
QDIFF18 63 0.333 0.223 1.4996 0.1398 
QDIFF19I 63 -0.063 0.207 -0.306 0.7603 
QDIFF2Z0 63 0.587 0.199 2.9496 0.0045 
QDIFF2Z1 63 0.317 0.110 2.874 0.0055 
QDIFF22 63 0.413 0.129 Ss. 193 0.0022 
QDIFF23 63 0.571 0.176 3.241 0.0019 
QDIFF2S 63 =0.397 0.189 =-2.103 0.0395 
QDIFF2Z5 63 -0.206 0.162 -1.275 0.2071 
QDIFF26 63 0.048 0.255 0.186 0.8527 
QDIFF27 63 -0.032 0.122 -0.261 0.7952 
QDIFF2Z8 63 0.397 0.162 2.4493 0.0174 
QDIFF29 63 -0.079 0.107 -0.743 0.4605 
QDIFF30 63 -0.413 0.139 2-94 3S 0.0042 
QDIFF31 63 -0.206 0.107 -1.937 0.0573 
ODILFF32 63 =-0 .032 0.068 -0.4968 0.6411 
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TABLE B-5. NAVIGATOR T-TEST RESULTS 


SFI IS SS POSIT=5 --- nce n nr nce nr rr rrr rr rrr re 
Variable N Mean Std Error T Prob>{T] 
QDIFF1 33 -0.515 0.302 -1.706 0.0976 
QDIFF2 33 0.242 0.261 0.928 0.3603 
QDIFF3 33 0.515 0.218 2.362 0.0244 
QDIFFG 35 -0.394 0.265 -1.489 0.1462 
QDIFF5 5 -0.758 0.292 -2.594 0.0142 
QDIFF6 33 0.727 0.227 3.200 0.0031 
QDIFF? 33 0.000 0.138 0.000 1.0000 
QDIFF8 5355 0.394 0.179 2.199 0.0352 
QDIFFI 33 -0.576 0.246 -2.338 0.0258 
QDIFF1O 33 0.355 0.198 1.685 0.1017 
QDIFF1l1 33 -0.636 0.245 -2.592 0.0143 
QDIFF12 35 0.515 0.175 2.948 0.0059 
QDIFF13 33 0.455 0.279 1.629 0.1130 
QDIFF14 32 1.438 0.269 5.340 0.0001 
QDIFF15 32 -0.03) 0.349 -0.089 0.9293 
QDIFF16 32 0.250 0.236 1.052 0.3008 
QDIFF17 33 -0.364 0.199 -1.831 0.0764 
QDIFF18 33 0.364 0.194 1.877 0.0697 
QDIFF19 33 -0.545 0.185 -2.947 0.0059 
QDIFF20 33 0.970 0.211 G.598 0.0001 
QDIFF21 33 0.152 0. 136 1.094 0.2820 
QDIFF22 a5 0.576 0.180 3.206 0.0030 
QDIFF23 33 0.455 0.200 2.274 0.0298 
QDIFF24 33 =O 275 0.239 -1.139 0.2632 
QDIFF25 33 -0.09] 0.262 -0.346 0.7313 
QDIFF26 33 -0.424 0.282 -1.504 0.1425 
QDIFF27 35 0.182 0.119 1.530 0.1358 
QDIFF28 33 0.182 0.160 1.139 0.2632 
QDIFF29 33 -0.333 0.193 =] .727% 0.0938 
QDIFF30 33 -0.182 0.119 -~1.530 0.1358 
QDIFF31 33 -0.576 0.238 -2.414 0.0217 
QDIFF32 33 0.091 0.110 0.828 0.4138 


62 


TABLE B-6. TACTICAL COORDINATOR T-TEST RESULTS 


i a a a a ee ee = - POSIT=6 sore rrrrcrrrr tt rrrrrccccrcc 
Variable N Mean Std Error T Prob>|T| 
QDIFF1 66 -1.076 0.166 -6.4969 0.0001] 
QDIFF2 66 0.12] 0.137 0.882 0.3810 
QDIFF3 66 0.591 0.1496 G.036 0.0001 
QDIFFS 66 0.030 0.208 0.196 0.88497 
QDIFFS5 66 =. 756 0.261 ~3.015 0.0037 
QDIFF6 66 0.333 0.115 2.900 0.0051 
QDIFF7 66 0.030 0.081] 0.375 0.7085 
QDIFF8 66 0.500 0.117 G.282 0.0001 
QDIFF9 66 -0.697 0.153 -G.566 0.0001 
QDIFF1O 66 0.364 0.139 2.610 0.0112 
QDIFF11 66 —O.7/735 0.206 -3.756 0.0004 
QDIFF12 66 0.273 0.125 2.181 0.0328 
QDIFF13 66 0.303 0.189 1.600 0.1145 
QDIFF14 64 1.141 0.215 5.314 0.0001 
QDIFF1L5 64 -0.39]1 0.215 ~1.813 0.07496 
QDIFF16 65 0.308 0.095 3.226 0.0020 
QDIFF17 66 — | 01 a ate 0.1497 ~1 .851 0.0687 
QDIFF18 66 -0.106 0.132 -0.806 0.49231 
QDIFF19 66 0.06] 0.158 0.382 0.7034 
QDIFF20 66 0.939 0.210 G.4967 0.0001 
QDIFF21 66 -0.030 0.103 -0.293 0.7706 
QDIFF22 66 0.258 0.139 1.856 0.0680 
QDIFF23 66 0.303 0.205 1.480 0.14938 
QDIFF24 66 -0.773 0.208 = 535 (ES 0.0004 
QDIFF25 65 -0.49496 0.177 -2.523 0.0141 
QDIFF26 65 -0.169 0.210 =O. oU07 0.49229 
QDIFF27 66 0.030 0.099 0.306 0.7602 
QDIFF28 66 0.303 0.114 2.654 0.0100 
QDIFF29 66 -0.4939 0.1493 -3.063 0.0032 
QDIFF30 65 -0.4946 0.150 -2.967 0.0042 
QDIFF31 66 -0.576 0.160 -3.594 0.0006 
QDIFF32 66 -0.061] 0.091 -0.664 0.5091 
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TABLE B-7. SENSOR ONE T-TEST RESULTS 


<= Se SS = SS = ae oe oD ae POSIT =? errr rrr rrr rn nn nner 
Variable N Mean Std Error T Prob>I{T| 
QDIFF1 60 -0.467 0.200 =2.35359 0.0228 
QDIFF2 60 0.267 0.176 55 0.1352 
QDIFF3S 60 0.400 0.153 2.622 0.0111 
QDIFFG 59 -0.169 0.222 -0.764 0.4478 
QDIFFS5 60 =0., 917 0.244 =3.756 0.0004 
QDIFF6 60 0.233 0.139 1.675 0.0993 
QDIFF? 60 0.050 0.069 0.425 0.4715 
QDIFF8 60 0.417 0.129 3.239 0.0020 
QDIFFY 60 -0.617 0.158 -3.907 0.0002 
QDIFF1O 60 0.200 0.106 1.891 0.0635 
QDIFFII 60 -1.083 0.2735 -3.963 0.0002 
QDIFFI2 60 0.150 0.164 0.913 0.3652 
QDIFF13 60 =0.017 0.191 -0.087 0.9307 
QDIFF1G 58 0.931 0.229 G.062 0.0002 
QDIFF15 58 0.328 0.288 1.136 0.2606 
QDIFF16 58 0.569 0.183 3.116 0.0029 
QDIFF17 60 -0.100 0.148 -0.676 0.5015 
QDIFF18 60 0.200 0.163 1.230 0.2236 
QDIFF19 60 —0 297 0.25949 -0.460 0.6475 
QDIFF20 60 0.417 0.221 1.887 0.0641 
QDIFF21 60 0.167 0.109 1.524 0.1328 
QDIFF22 60 0.300 0.151 1.988 0.0514 
QDIFF23 60 0.167 0.149 1.120 0.2671 
QDIFF24 60 -0.500 0.205 -2.437 0.0178 
QDIFF25 60 0.017 0.188 0.089 0.9296 
QDIFF26 60 =0.133 0.224 -0.596 0.5532 
QDIFF27 60 0.183 0.188 0.976 0.3332 
QDIFF2Z8 59 0.186 0.158 1.183 0.2415 
QDIFF29 60 0 o> O.177 -1.880 0.0651 
QDIFF30 60 =) 21635 0.188 -0.710 0.4807 
QDIFF31 60 =) 235 0.178 -1.308 0.1961 
QDIFF32 60 =0.017 0.061 =e 0.784] 
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TABLE B-8. SENSOR TWO T-TEST RESULTS 


7 —— POSITS8 errr rrr rrr rrr rr rrr ttre: 
Variable N Mean Std Error T Prob>ITI 
QDIFF1 25 -0.680 0.309 “2.198 0.0379 
QDIFF2 25 0.440 0.265 1.660 0.1099 
QDIFF3 25 0.200 0.216 0.926 0.3638 
QDIFFS 25 0.400 0.306 1.309 0.2028 
QDIFF5 25 -0.960 0.381] =2.522 0.0187 
QDIFF6 25 0.480 0.209 2.295 0.0308 
QDIFF? 25 0.080 0.080 1.000 0.3273 
QDIFF8 25 0.440 0.327 1.3496 0.1910 
QDIFF9 24 -0.500 0.376 eo U 0.1965 
QDIFF1O 25 0.520 0.239 2e1i7 0.0396 
QDIFFl]1 25 -] .080 0.404 “2.674 0.0133 
QDIFFI2 25 -0.240 Ue 273 =(.00.) 0.38/72 
QDIFF13 25 -0.640 0.336 -] .904 0.0689 
QDIFF14 bes 1.043 0.277 3.76] 0.0011 
QDIFF15 ZS 0.174 0.411 0.4924 0.6760 
QDIFF16 23 0.391 0.306 1.277 0.21498 
QDIFF17 25 =O 5c U 0.269 -1.189 0.24960 
QDIFF18 25 0.2490 0.3498 0.690 0.4967 
QDIFF19 25 -0.640 0.282 -2.268 0.0326 
QDIFFZ0 25 0.720 0.286 2.518 0.0189 
QDIFF21 25 0.080 0.114 0.700 0.4907 
QDIFF22 25 0.600 0.337 782 0.0874 
QDIFF23 25 0.160 0.229 0.700 0.49907 
QDIFF24 25 -Q0.720 0.354 -2.036 0.0529 
QDIFF25 25 -0.440 Heo 00 -1.4964 0.1560 
QDIFF26 25 -0.040 0.372 -0.108 0.9152 
QDIFF27 25 -Q0.320 0.269 -1.189 0.24960 
QDIFF28 24 0.000 0.190 0.000 1.0000 
QDIFF29 25 -0.080 0.258 -0.310 0.7589 
QDIFF30 25 0.280 0.363 0.77] 0.4480 
QDIFF31 25 -0.400 0.163 -2.449 0.0220 
QDIFF32 25 -0.120 0.133 -0.901] 0.3765 
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TABLE B-9. SENSOR THREE T-TEST RESULTS 


(<== = = = 3S ee ee POSIT=9 errr errr rt tr rrr er rrr rs rrrsrr-: 
Variable N Mean Std Error T Prob>[{TI 
QDIFF1 56 -0.66] 0.179 -3.694 0.0005 
QDIFF2 56 0.193 0.1493 1.000 0.3217 
QDIFFS3 56 0.107 0.136 0.765 0.9357 
QDIFFS 56 =) 25> 0.191 al. O55 0.5151 
QDIFFS5 56 -0.518 0.291 Toc 0.0802 
QDIFF6 56 Oe1Z5 Up FE | 1.124 0.2661 
QDIFF?7 56 02125 0.085 1.4975 0.1960 
QDIFF8 56 0.518 0.119 G.334 0.0001 
QDIFF9 56 -0.482 0.180 -2.676 0.0098 
QDIFF10 56 0.196 0.112 1.749 0.0858 
QDIFF11 56 -1.339 0.255 -5.248 0.0001 
QDIFF12 56 0.252 0.1496 1.586 0.1185 
QDIFF13 56 0.232 0.213 1.089 0.2809 
QDIFF14 54 1.241 OF 215 5.780 0.0001 
QDIFF15 54 0.111 0.260 0.4928 0.6707 
QDIFF16 54 0.389 0.148 2.625 0.0113 
QDIFF17 56 -0.357 0.191 -1.866 0.0674 
QDIFF18 56 =0.107 0.168 -0.636 0.5273 
QDIFF19 56 -0.018 0.201 -0.089 0.9297 
QDIFF20 56 0.518 0.213 2.929 0.0184 
QDIFF21 56 0.179 0.092 1.936 0.0581 
QDIFF22 56 0.554 0.169 3.278 0.0018 
QDIFF23 55 0.309 0.158 1.962 0.0550 
QDIFF2S4 56 -0.661 0.214 -3.093 0.0031 
QDIFF25 56 -0.054 0.177 -0.302 0.7637 
QDIFF26 55 -0.309 0.223 -1.386 0.1715 
QDIFF27 56 -0.036 0.238 -0.150 0.881] 
QDIFF28 56 0.252 0.210 1.105 0.27490 
QDIFF29 56 -0.411 0.194 -2.859 0.0060 
QDIFF30 56 -0.268 0.209 -1.280 0.2061 
QDIFF31 56 -0.161 0.157 -1.026 0.3093 
QDIFF32 56 0.036 0.102 O55! 0:7 271 
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TABLE B-10. INFLIGHT TECHNICIAN T-TEST RESULTS 


ee POSIT=#=10 errr r rer nr nr rr nn rrr ne 
Variable N Mean Std Error T Prob>|T| 
QDIFF1 G0 = O75 0.205 -1.832 0.07496 
QDIFF2 G0 0.675 0.225 3.004 0.00496 
QDIFF3 G0 0.500 0.152 3.291 0.002] 
QDIFFS G0 -0.450 0.253 -1.778 0.0832 
QDIFF5 G0 -1.000 02275 -3.636 0.0008 
QDIFF6 G0 02275 0.160 1.718 0.0937 
QDIFF7 G0 02225 0.082 2533 0.1334 
QDIFF8 G0 O.725 0.183 3.972 0.0003 
QDIFF9I G0 -0.49425 0.1493 -2.978 0.0050 
QDIFF1O0 G0 0.000 0.062 0.000 1.0000 
QDIFF11 40 -1.400 0.253 -5.54] 0.000] 
QDIFF12 G0 0.450 0.152 2.966 0.005] 
QDIFF13 G0 -0.050 0.193 -0.260 0.7966 
QDIFF1S 37 1.027 0.304 3.380 0.0018 
QDIFF15 37 0.270 0.341 0.792 0.4338 
QDIFF16 37 0.4932 0.207 2.089 0.0438 
QDIFF17 G0 O27 5 0.202 -0.866 0.3920 
QDIFF18 G0 -0.025 0.216 -0.116 0.9086 
QDIFFI19 G0 -0.325 0.249 -].305 0.1996 
QDIFF20 G0 0.400 0.267 1.4996 0.14928 
QDIFF21 G0 0.375 0.181 2.066 0.0455 
QDIFF22 G0 0.525 0.193 Cte 0.0096 
QDIFF23 39 ae | 0.193 1.199 0.2381 
QDIFF24 G0 =O2175 0.211 -0.827 0.4130 
QDIFF2Z5 G0 -0.025 0.201 -0.124 0.9016 
QDIFF26 38 -0.105 0.304 -0.347 0.7308 
QDIFF27 G0 U7 225 0.222 =1.013 0.3173 
QDIFF28 G0 0.125 0.125 1.000 0.3235 
QDIFF29 G0 -0.750 0.174 -G.298 0.000] 
QDIFF30 G0 =()- 57 5 0.196 i ES 0.0055 
QDIFF31 40 Umea) 5 0.228 -1.642 0.1087 
QDIFF32 G0 0.150 0.057 22620 0.0124 
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TABLE B-11. ORDNANCE T-TEST RESULTS 


ee POSITHl] erent nmr emer n renee eee ene 
Variable N Mean Std Error T Prob>|{T| 
QDIFF1l 37 -0.946 0.245 -3.862 0.0004 
QDIFF2 37 0.622 0.224 2.778 0.0086 
QDIFF3 37 0.568 0.2494 2.329 0.0256 
QDIFFS 36 -0.056 0.316 -0.176 0.8615 
QDIFF5 37 -1.216 0.320 -3.802 0.0005 
QDIFF6 37 0.081 0.206 0.393 0.6968 
QDIFF7 36 -0.083 0.083 -1.000 0.3242 
QDIFF8 37 0.784 0.266 2.9496 0.0056 
QDIFF9 37 0.000 0.164 0.000 1.0000 
QDIFF10 37 -0.027 0.137 -0.197 0.8446 
QDIFF11 37 -1.243 0.441 -2.820 0.0078 
QDIFF1l2 37 -0.027 0.147 -0.183 0.8556 
QDIFF13 37 -0.216 0.174 -1.2494 0.2217 
QDIFF14 36 1.306 0.245 5.329 0.0001 
QDIFF15 36 -0.472 0.327 -1.443 0.1580 
QDIFF16 36 0.722 0.176 G.093 0.0002 
QDIFF17 S7 -0.216 0.260 -0.831 0.4117 
QDIFF18 37 0.108 0.238 0.454 0.6526 
QDIFF19 37 -0.324 0.280 -1.160 Oe o7 
QDIFF20 37 0.919 0.220 G.168 0.0002 
QDIFF21 37 0.243 0.131 1.859 0.0713 
QDIFF22 37 0.486 0.143 3.402 0.0017 
QDIFF23 37 0.081 0.136 0.595 0.5557 
QDIFF2S4 37 -0.432 0.291 -1.4984 0.1465 
QDIFF25 37 -0.297 0.208 -1.428 0.1619 
QDIFF26 37 -0.162 0.289 -0.562 0.5778 
QDIFF27 o7 -0.378 0.230 -1.642 0.1092 
QDIFF28 37 0.730 0.231 3.154 0.0032 
QDIFF29 37 -0.378 Oe252 -1.500 0.1422 
QDIFF30 37 -0.324 O27 1 “1.195 0.2399 
QDIFF31 37 -0.432 0.321 -1.348 0.1861 
QDIFF32 37 -0.054 0.116 -0.466 0.6437 
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